“Teleoperation” or “telepresence operational control” generally refers to the manipulation of physical objects by an operator from a distance. Among other things, teleoperation allows an operator to use a remote implement, such as an unmanned vehicle, to perform a task using a human-computer interface (HCI). Teleoperation advantageously allows an operator to implement remote tasks in hostile or inaccessible environments, where human exposure might be hazardous but human intelligence is nonetheless required.
A number of HCIs exist, that will allow a human operator to operate a remote vehicle, such as an aerial drone. Some particular HCIs respond to motion of the human body. Generally, these systems monitor spatial information to detect motion of given parts of the human body, such as the hands, arms, legs, or eyes, associate specific body motions with particular vehicle operational commands, and then generate those commands for controlling the remote vehicle.
Another more specific form of HCI is the brain-computer interface (BCI). BCI generally senses electrical brain activity and translates that brain activity into computer commands. One approach uses electroencephalogram (EEG) data captured using a standard sensor cap, similar to a swimming cap. Another approach uses an electromagneticgram (EMG) to detect brain activity. In both cases, off-the-shelf hardware is available, which is relatively inexpensive and non-invasive.
However, BCI is still subject to some significant disadvantages, and currently insufficient alone to support control of a remote vehicle. Among other things, the human operator must maintain concentration to maintain vehicle control and significant training of the system is required.
Augmented reality (AR) systems combine sensor information with computer generated information to generate an enhanced user interface. In particular, in the AR technique known as synthetic imagery, computer-generated information, such as images and text, overlays real-time information provided by a sensor. For example, terrain information may overlay a live video feed from a vehicle to allow remote operator to navigate that terrain with a reduced risk of collision. In addition, AR systems can be immersive, for example using a hologram, which allows the user to experience the environment in which the sensor is operating.
Currently, no interface exists that combines biometric data (i.e., HCI data from body movements and BCI) with an AR holographic display device and allows an operator to remotely control an unmanned vehicle (UV), such as an unmanned aerial vehicle (UAV).
Unmanned vehicles have numerous applications in the law enforcement and military fields; however, while they have been successfully used to avoid dangerous confrontations and locate weapons and contraband in open environments, they are still subject to some serious limitations. For example, an unmanned vehicle, such as an aerial drone, may allow law enforcement officers to safely reconnoiter the outside of a building, but not the identification of individuals and object within the building. As a result, officers are often unable to determine whether entry into a building or other structure presents a significant personal risk, whether they have probable cause to enter the building immediately, and/or whether they have sufficient information to request a search warrant.
In order to achieve full autonomy, an unmanned vehicle typically needs to possess the ability to explore its environment without user-intervention, build a reliable map of the environment, and localize itself within the map. Simultaneous Localization and Mapping (SLAM) advantageously allows an operator to interact in complex or difficult to navigate environments by reducing or eliminating the piloting burden.
Cycle counting is the process of auditing inventory at distribution centers or warehouses. In a cycle count, a section of the inventory in a specific location of a warehouse is counted to measure inventory accuracy relative to the records in the warehouse management system (WMS). Contrary to physical inventory, cycle counting requires only the section being cycle counted to be measured, as opposed to physical inventory where the full inventory of the warehouse is measured during a time in which the operation of the warehouse would be stopped.
Computerized object identification and recognition generally refers to the ability for a computer system to be able to match objects against a corpus of objects previously provided. Among other things, computerized object identification and recognition advantageously allows an operator to label a large number of objects without requiring their intervention.
Currently no system exists that combines SLAM with computerized object recognition and identification that allows one or more remotely controlled UAVs to perform inventory cycle counting.